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Abstract 

Let C be a real- valued M x M matrix with singular values Ai > ... > Am and E 
a random matrix of centered i.i.d. entries with finite fourth moment. In this paper 
we give a universal upper bound on the expectation of ||7r r .X'||j| 3 — UtiyXH;^, where 
X :— C + E and 7r r (resp. n r ) is a rank-r projection maximizing the Hilbert-Schmidt 
norm ||fr r .X||s 2 (resp. ||7rrC||s 2 ) over the set Sm,t of all orthogonal rank-r proj ections . 
This result is a generalization of a theorem for Gaussian matrices due to Rohde [?2012)l . 
Our approach differs substantially from the techniques of the mentioned article. We 
analyze |7r r X||| 2 — ||7r r X||| from a rather deterministic point of view by an upper 
bound on ||7r r X||| 2 — | 7r r X | \ s , whose randomness is totally determined by the largest 
singular value of E. 

1 Introduction 

Let C be a real- valued M x M matrix, M € IN, with singular values A^ = Afc(C), k = 
1, M, in decreasing order and E a M x M random matrix, whose entries are centered 
i.i.d. real- valued random variables. We denote the singular values of E by o\ > ... > a m- 
Further let ir r be a rank-r projection, which maximizes the Hilbert-Schmidt norm ||7TrC||s 2 
over the set Sm,t of all orthogonal rank-r projections into subspaces of R M . 
Consider the process (Z^^^^Sm r defined by 

Z* r : =||7f r X||| 2 -||7r r X||| 2 , X:=C + E, (1.1) 



and its supremum denoted by 



Z* T = sup Z #r , (1.2) 



where 7r r is a location of the supremum. In general, 7r r is not unique, since the distribution 
of the entries is allowed to have mass points. 



Rohde 1(2012)1 investigates the accuracy for empirical reduced-rank projection of the deter- 



ministic matrix C perturbed by a Gaussian noise matrix E. More precisely, she derives 
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upper and lower bounds on ~EZ^ r . The proofs in the mentioned article rely heavily on 
the Gaussian distribution of E. In particular, th e main ingredients for the upper bound 
are among others S^-S'oo-chaining and the Borell \( 1975)\ - Sudakov and Tsirel'son IC19741I 
inequality. Since Z is not centered, the clue of the paper is a slicing argument for Sm,t 
to proceed to centered Gaussian processes on well-chosen slices. Beyond, for the proofs of 
lower bounds on EZj r the invariance property of the distribution of E under orthogonal 
transformation and Sudakov's minoration are used. Due to the dependence of the proofs 
on the Gaussian distribution, naturally the question arises, whether the results of Rohdc 



f2012 h hold for a larger class of probability distributions of the independent entries Ei 



Before we pu rsue this question, we first recapitulate the upper and lower bounds from 



Rohde [2012] 



Theorem 1 (Upper bound for Gaussian matrices) 

Under the former assumptions and notations let the distribution of Eij be centered Gaussian 
with variance a 2 and rank(C) > r. Then in case of r < M — r the following bound holds 



EZ* r <a 2 rM[mm(^,l< Al 



1 V^2r \2\ 2 , , 2 



A 2 . I 0~\/ AI ~ 



(1.3) 



A 

where y2 _i'i — is set to infinity, if X r = A r +i. 



Theorem 2 (Lower bounds for Gaussian matrices) 

Let E^ be centered Gaussian with variance a 2 . 

(i) Let Ai = ... = Am = a, then 

EZ* r >E( sup \\n r E\\ 2 2 - \\n r E\\ 2 S2 ) (1.4) 

and for r < M — r 

WjZ ~ 

liminf — > crrv M — r. (1-5) 

a— >oo (y 

(ii) Denote 

Zi s ■= sup ||7r 5 (C a ,„ + E) ||| 2 - ||tt s (C QiS + E) ||| 2 , 1 < s < M, 

where the singular value decomposition of C a%8 is given by UaId s V , a > 0. Then it holds 

liminf max EZ| >a 2 r(M-r). (1.6) 

a-s-oo S £{r,M~r} 3 
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(Hi) Let r=l. There exists an Mq € IN, so that for all a > and any M > Mg ?£ ZioZcfe 



n ^fxM^*— E SUP H^lls a -|Mli • (1-7) 



(11.41) . (jl.6l) and (|1.7p indicate, that there docs not exist a more favorable matrix than C = 
in terms of accuracy of ||7r r X||s 2 for ||7r r X||5 2 . For r = 1 this statement is proven. (|1.5|) 
shows, that in general the upper bound a 2 rAf(l H — ^Jy) is unimprovable. Nevertheless it 
is possible to state a more refined upper bound, as seen in Theorem 1. 
In this article we generalize Theorem 1 to all random matrices of centered Lid, en tries 
with finite fourth moment. Our approach differs significantly from Rohdc 1(2012)1 The 



key argument is an upper boun d on Z* , whose randomness is totally determined by G\. 



Concerning the results of Latala 1(2005)1 our generalization depends on both the variance 
and the fourth moments of the entries, since o\ does. In a broad sense we exploit the 
location 7r r of the supremum of the process Z. The clue is, that given oi, Z attains its 
supremum on a rather small S^-ball depending on &\. Our upper bound on Z% T takes this 
into account. 

The main result of this article is the following: 
Theorem 3 (Universal upper bound) 

Assume that the entries Eij of the random matrix E have finite variance a 2 and finite 
fourth moment 7714. In this case the following inequality holds 

EZfi r < r(M - r) min(I, II, III), (1.8) 



where 



(a + ym~l) , (1.9a) 

v .-„ ifK>\r + U (19b) 

if -V = A r +i, 



III 




^2 Y2 2r A 2 

r(Ai-?)k ' ^ + tf K > ' (1.9c) 

if X r = 0. 



This result is a generalization of Theorem 5.1 of Rohde (2012)1 (resp. Theorem 1 stated 
above). We give a brief discussion of this fact later. 

The article is structured as follows. In the next section we introduce further notations. We 
give some elementary estimations on traces of certain matrices in the third section. Most 
of the results in this section are stated for deterministic matrices. In the fourth section a 
proof of Theorem 3 is given. Finally in the last section we give a further application of 
Proposition 1 of section 3. We derive intervals containing liminf m-kx> X\(Cm + Em) and 
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lim supj^^oo Ai (Cm + Em) almost surely, where Cm is a deterministic M x M matrix and 
Em is a M x M random matrix of i.i.d entries with variance cr 2 M _1 . 

2 Preliminaries 

We split Z into two subprocesses Z 1 and Z 2 given by 

Z\ r := \\w r C\\ 2 S2 - ll7iy.CH!, + 2tr(£ T (ir r - n r )C), 
Zl := \\w r E\\ 2 S2 - \\7r r E\\ 2 S2 . 

So it holds Z = Z 1 + Z 2 . Further we denote by Tr* a location of the supremum of Z 1 . 
A < B means, that A is equal or less than B up to some universal constant. If A < B 
and B < A, we write A ~ B. We denote the Schatten-p-norm, 1 < p < 00, on R MxM by 
|| • \ \s . For C G R MxA/ with singular values Ai > ... > Am the Schatten-p-norm of C is 
given by 

M 

^ Af for 1 <p < 00 and \\C\\ Soo = Ai. 
i=i 

In particular we will use the Hilbcrt-Schmidt norm || • \ \g 2 and the spectral norm || • Ws^- 
For the trace of a matrix C G R MxM we write tr(C) and for its transpose C T . Moreover 
put A r := X)ilr+i an d := r A (ill — r). The Euclidean sphere is denoted by S^ -1 . 
For any set £> C Sm.t wc define £? := Sm.t \ B. Lastly, [^J is the smallest integer larger 
than x G R. 

3 Estimation of traces involving differences of projec- 
tion matrices 

In this section we derive estimations of traces of certain matrices like those arising in the 
process Z. However, the results are stated in a quite general way and are phrased in a 
deterministic setting. 

First recall some basic properties of orthogonal projections. By definition we have 

7r r = ttJ and 7iy = 7r r 7r r for n r G Sm,t- 

Therefore every orthogonal projection 7r r is positive-semidefinite. For 7Tr ,71"^ G Sm.t 
with eigendecomposition 7Tr = UId r U T and 7Tr ^ = UId r U T we get 

tr^V^) = tr (U I d r U T U I d r U T ) = tr (U T U I d r U T U I d r ). 

The matrix P := U T UId r U T U is also an orthogonal projection. Since P is positive- 
semidefinite, the diagonal entries of P are nonnegative. It follows 

r 

tr(7r«7r( 2) ) = ti(PId r ) =J2 P » ^ °- 

i=l 
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We conclude 

\\4 2) - tt^Hs, - \\(Id-4^) - (Id - 7T^)\\ S2 < V2^7. 
Finally, note that by symmetry of 7Tr ^ — tt^- 1 ' 1 we have 

\\4 2) -4 1) \\s ae = sup I^Ctt^ -7r«)x| 

= sup | x T 4 2 ^ x - x t tt^x I < 1. 
e[o,i] e[o,i] 

The next lemma provides a useful estimate to bound tr(E T ( : K r — 7r r )C) and Z\ . 
Lemma 1 

Let 4 G Sni,r and A, B G R MxM , then the following inequality holds 



tr(A T (4 2 ^ -4^)B) < V2^\\A\\ 3 J\B\\sJ\4 2) '4^1 



s 2 • 



Proof. First, note that 

7r( 2 ) - ttC 1 ) = tt^ - Tr^TT^ + Tr^TT^ - 7T« = 7r< 2 > (Jd - TT <^ ) + (tT^ - Id)l 

From the proof of Proposition 8.1 in Rohde (2012) we get 

\\(id-4 2) )4 1) U = \\4 2) (id-4 iy )\\s 2 = -4 2) 



, J{ r 7T r 1 1 Si ■ 



By Cauchy-Schwarz inequality follows 



tr(A T (^ 2) -4^)B) 

= tr(A T 4 2 \ld- 4^)B) + tr(A T (^ 2 > - Id)4^B) 

< ( 1 1 S^ T 4 2 ) ||s 2 A \\(Id- tt« )BA T \\ S2 )\ \nW {Id _ ^i) ) 1 1 



+ fllTr^S^Ha A \\BA T (Id-4^)\\s 2 ) \\(4 2) -Id)4 1] \\ 



\s 2 



< -L^^/\\BATABT\\ S J\4 1] ~ 4 2) \\s 2 



-j=y/^,J\\BATABT\\ s J\4V -4 2) \\s 2 



<V^\A\\sJ\B\\ 3 J\4»-irW\\ 8a 
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The statement of the lemma is optimal to the effect, that in case of M > 2r, orthonormal 
vectors u\, u r , u\, u r and matrices 



4^ = y^Ujuf , ir^ = ^(yl - a 2 Uj + aui){y 1 - a 2 u, t + aui) T 



1=1 



A = /ild, B = v (ttW - 7r< 2; 



the left-hand side of the inequality attains the upper bound for any real numbers 
< a < 1, fj,, v > 0. We give a brief computation: 



tr(A T (7rW - 4 2) )B) = fiutv tt^ - ]T( y/l~a^u t + a.Ui)(\Tl - o?u t + au t ) T 



Tt!f}' — V^CV 1 Q 2 Mi + aiti)(\/ 1 — a 2 iti + aii 



= /j,u (V-2tr (ttW^)) 
= [iv (2r - 2r(l - a 2 )) 



2r[iva v2r. 

So it remains to show, that ||7ir — tt-t 2 '' 1 1 s 2 = o^\/2r and 1 1 -zr^- 1 ^ — 7Tr Hs,*, = a - 
The first equation is obvious concerning the previous calculation, since 



(i) a) <2) 



4 1] ~ 4 2} \\S 2 = \/tr [ [ TTf - 7T.r ) - 7T. 



aV2r. 



To prove the second equation, one can check, that a and —a are the only non-zero eigen- 
values of iTr — 4^ and their eigenspaces are given by 



and 



W a = span ■ 



W- a = span 



l + a 



1 — a 



1-a 



l + a 



i = l,...,r 



Since 4^ — ttt is symmetric, it holds 



\4 1] - 4 2) \\s~ = m^(x max (4^ - 4^), \\ m u4 1] - 4 2) )\) = «• 



As Z has a negative drift, Lemma 1 is not useful to bound H^rCI!?^ — | |7T r C| ]^ . Therefore 
the next lemma gives an estimate on the drift term. It is significant for our subsequent 
computations, that the distance ||7ry — 7r r ||s 2 influences the drift term rather squared than 
linearly. 
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Lemma 2 

(i) For any jt r £ Sm,t the following inequality holds 

W^rC\\% 2 - ||7r r C||5 2 < -- (A^ - Xl +1 ) \\n r - ir r \\%- (3-3) 

(ii) For any fry g Sm,t! such that \\% r — 7r r ||s a > X~ 1 y/2A r , we have 



|fr r C||| 2 - ||7r r C||| a < --\ 2 r \\*r - 7r r ||| a + A r . (3.4) 



Proof. Both inequalities are clear for A r = 0. Hence assume A r > 0. 

(i): It is obvious in case of A r = A r+ i or 7r r = 7r r , so assume moreov er A r > A r+ i and 



7ty 7^ 7r r . Now (i) follows easily from Proposition 8.1 in Rohde 1(2012)1 combined with an 
cpsilon of room argument. Let e > and set 



\\n r - 7Tr||s 2 > \\^r - K r \\s 2 - £ = \ -72 ^2J- 

y A r — A r+1 

Then from statement (8.1) in the mentioned proposition we get 

|Kq|| 2 - \\n r C\\% >5= 1 -{\ 2 r - X 2 r+1 ) (\\7Tr - if r \\s 2 - ef . 

By multiplication by -1 and taking the limit e — > (i) is proved. 

(ii): This part follows analogous to (i). □ 

Now we derive an upper bound on Zl x , which will be useful to estimate the expectation 
of Zft T ■ In a certain way the upper bound regards the location of k\ . 



Proposition 1 

For the supremum Zh of the process Z 1 we have Zh < Y with 

Y := min (L, II', III') , (3.5) 

where 

I' :=4r M XnTi, (3.6a) 

{A 2 
M K-K+i 1 J r+u (3.6b) 

oo if X r — A r +i, 

m , = | max (47^72;^!, 8r M §^) if X r > 0, 
I 00 if X r = 0. 
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Proof. We prove for I', II' and III', that Z\ x is less or equal to each one. 

Z\i < I' : Since ||7ryC||| 2 — IKrC||| 2 < 0, we get by Lemma 1 for any fry € Sm,t 

Z\ t < 2y/2rM<r 1 \ 1 \\w r - ir r \\s 2 < 4r A/ CTiAi. (3.7) 
As 4r A /criAi does not depend on 7r r , < I' holds. 

Z\i < II' : Assume A r > A. r+ i. We obtain by Lemma 1 and 2(i) for any tx t £ Sm,t 
Z\ r = ||fr r C||| 2 - ||7r r C||| 2 + 2tr(£ T (^ r - n r )C) 

^ _ 2 v r ~ V+l) ll^r — ""Villa + 2 sF^FmOx \\ \ \n r — TT r \\s 2 



Then maximizing the right-hand side of the equation 

Z\ t < (X^, — A^ +1 ) ||7iy — 7r r ||| 2 + 2yj2r M a 1 \i\\ : K r — 7r r ||s 2 
over all 7r r € Sm,t provides the claim. 

Z\i < Hi' : Assume A r > 0. In order to prove the last bound we split Sm,t into two sets 
and take the supremum of Z 1 on this sets separately. We define 



Bur := {tt t € Sm,t ■ \\^r ~ ' K r\\s 2 < K 1 \/2A J .}. (3.8) 

It holds 

Zii=max( sup 2i r , sup Z\\ . (3.9) 
For the first expression in the maximum of (|3.9[) we get analogous to the proof of Z\ x < I': 
sup Z\ < sup 2v / 2rj\7c r iAi||7r r — 7r r ||s 2 < 4v / TA/A^-r-o'i. (3.10) 

i r eB UI , T n r £B m , A r 



It remains to bound the second expression in the maximum of (|3.9[) . By Lemma 1 again 
and by Lemma 2(h) follows for any 7r r G -Bur 

Z\ r < -^Iftr ~ 7T r ||! 2 + A r + 2V2n^<7i Ai ||7T r - 7T r ||<? 2 . (3.11) 
The right-hand side attains its global maximum on 

{ir r e S M>r ■ \\n r - 7Tr||s 2 = 2 v / 2r^7(Ti^| A ^2r M }- (3-12) 

If 2V2rI7cri^ < then it holds 

{7tv G 5M,r : ll^r - 7Tr||s 2 = 2 v / 2T^cri^i A \J2t m } H Bin' = 0. 
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In this case due to reasons of monotonicity the right-hand side of (|3.11[) restricted to 
{w T G Sm,t ■ 1 1 Try — 7iv||s2 — A~ 1 \/2A r } attains its maximum on {ir r G Sm <t : | JtTt- — 7T r ||s 2 = 
A^ 1 \/2A r }. So we have 



-Ai 

Otherwise by p,12j) follows 



t {V2K-r>2^2FH^ 1 } Z k ^ 1 { v ^A7>2 v ^7^ : a 1 } 4 ^ r M A r^-^l- (3-13) 



/ A 2 

< hv^<2V^^} 8r M^*l (3.14) 



By (|3~T3)) and (pTT^ we get 

sup = sup l {yls - >2V5 _A i(Ti} Zi r + l { ^_^ 2V5 _A iCTi} Zi r 

Ai , n A 2 



^ 1 {v^A7>2V2f^^ : a 1 } 4 V /r MA ? .^-(7 1 + 1 { ^-^^—^ } 8r M ^<7? 



-Ai n A? 



< max \4y/r M Ary(Ti, %r M ^lj ■ (3.15) 
Finally combining (|3~TU)) and (j3~T5|) yields Z^ < III'. □ 



4 Proof of Theorem 3 



Now we are ready to prove Theorem 3. We first state a result by Latala l(2005 M in simplified 
terms. 



Theorem 4 (Latala [{2005)|) 

For any random matrix E of centered i.i.d. entries with variance a 2 and fourth moment 
mi the following inequality holds 

Ecr 2 < M (a 2 + v 7 ^) . (4.1) 



Note that the original result is phrased for the expectation of <j\ and not of af, but actually 
the proof includes statement (|4.ip under the additional assumption, that the distribution 
of the entries is symmetric. So one only has to adjust the symmetrization argument from 
E|[£?||,g 00 to E||£?||| . Therefore let E be an independent copy of E and e a M x M- 
random matrix of i.i.d. Rademacher variables, which is independent of E and E. Then we 
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obtain by Jensen's inequality for conditional expectation 
D<7 2 = EH^Hl^ 

= E ( sup x T Ey ) 

= E sup (x T Ey) 2 

= E sup (x T Ey - Ex T Ey) 2 

= E 



sup (e (x T Ey - x T Ey\E 

< E sup E ( (x T Ey - x T Ey] 2 \E 

< EE | sup (x T Ey - x T Ey) \e) 
^yes"- 1 v J J 

sup ( x T Ey — x T Ey 



= E 

x,y£S 

= E\\E-E\\ 2 Sao 
= n\e-(E-E)\\ 2 Sao 

<-E[\\£-E\\ Soo + \\£-E\\s a ' 

<E(2\\s-E\\ 2 Sx +2\\e-E\\ 2 c 

< 4E||e ■ ^Hl^, 

where • denotes the Hadamard product. Since Eij and SijE^j have the same variance and 
fourth moment, statement (|4.1|) holds without the symmetry assumption on the distribution 
of the entries. 

Proof of Theorem 3. Beforehand note, that by distinguishing the cases r < 4f and r > 4f 
it holds 

r(M - r) < r M M < 2r(M - r). 
Now we commence the proof of Theorem 3. First we get 

EZ #r <EZii+E sup Z| r . (4.2) 

Applying Theorem 4 to the second summand yields 

E sup Z\ r < 2r M Ea\ < r M M (a 2 

»r6S«,r 

< 



r(M - r) (a 2 + Vm^j . (4.3) 
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Therefore 

E sup Z\ r < r(M-r) min(I, II, III). (4.4) 
So it remains to prove, that 

EZ^ < r(M - r) min(I, II, III). (4.5) 
By Proposition 1, monotonicity of integral and Theorem 4 we get 
EZji < EF 

< min (EI', EII', EIII') 

< min 4r M AiEcri, 4r M — Ecr 2 , Emax 4 v / r^VT-oi, 8r A/ -±cr 2 

\ A,. — A r+1 \ A r A r J J 

/ ^2 A A 2 

< min 4r M AiE<7i, 4r A/ — ^Ect 2 , 4 A /r A/ A r -i]E ( T 1 + S^-fE^ 2 

\ A r 'V+l ^ r 

/a A 2 

< r(M - r) min — L (a + </mZ) , — ( ct2 + V^) > 



+^) + V r(M -r)A 2 

< r(M - r) min (I, II, III) . 

□ 

As men tioned in the introduction, this result is a generalization of Theorem 5.1 of Rohde 



f2012l To check this consider the case, where E is a Gaussian matrix and r < M — r. Since 
the fourth moment of a centered Gaussian random variable is given by 3<r , the right-hand 
side of inequality (|1.8[) may be rewritten as 



G 2 rM min I 1 



Ai A 2 A 2 / - 53»=r+l Ai 



aVU 7 A 2 -A 2 +1 ' A 2 V A 2 (jy/M 

where the constant in (JT7HJ) is now specific to Gaussian matrices. So we have to show, that 
/ Ai 

a rM min 



min 1 




oVm' A 2 - A 2 +1 
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This follows by (|4.6[) and (|4.7|) in the next computation 



I A? Al A , ■ ( r Si=r+1 Af Ai ^1 

a rM mm -r?' 1 



min 1 



Al / r Si=r+1 A| Ai 



O-VM V A r <T\/M' 



Af A| A? / r X)i=r+1 A| Ai 



A 2 . A 2 — A r , j A 2 , y A 2 , o\f~M 



Ai Af Af / - X);=r+i Af Ai 
< 2. rMmm | 1 + ^ + ^ ^ | (4.6) 



< 2a 2 rM min I 1 



= 2cr 2 rM min I 1 



Ai A? A 2 X 2 r Yli=r+1 X i Ai 



ay/M' A 2 A^-Af +1 ' A 2 V A 2 
A? 



< 2a 2 rM I min I —t , 1 




(4.7) 



The last line arises by the simple observation, that min(a, b + c) < min(a, b) + c for any 
aiet and c > 0. 



5 Application: Localizing the largest singular value of 
a deformed random matrix 

As a further application of Proposition 1 we take a classical view on random matrices. 
Hence, let (-Ey-^jgN be a doubly indexed sequence of centered i.i.d. random variables 
with variance a 2 and finite fourth moment. By (Em) we denote the sequence of M x M 
random matrices l/\/M(Eij)i t j<M- (Cm) is a sequence of deterministic M x M matrices. 
Assume furthermore that the first and the second singular values Ai (Cm) and A2 (Cm) 
converge to some real numbers Ai > A2 > for Al — > 00. We specify an interval containing 
liminfA/^oo Ai(Cm + Em) and limsupj^^^ Ai(C/\/ + Em) almost surely. 



Corollary 1 

Under the former notations and assumptions let (uii)?:g]N be a sequence of real numbers, 



Empirical projections of deformed random matrices 



13 



such that 

/ A/ 



{un,-,UMlf 



\i=i / 

is the left singular vector of Cm corresponding to the largest singular value. If there exist 
/3 > 1,0' > with 

Ei=i< M 

then it holds a.s. 



A 2 +a 2 <liminf X 1 (C M +E M ) 

M—too 



< limsup Ai(C M + £ M ) < \ Xf + 4er 2 + 16a 2 A ' (5.2) 
m->oo y Aj — A 2 



Thus, if in the large amplitude regime the values Ai and A2 are well-separated, then the 
largest singular value of Cm + Em is typically close to Ai but lar ger. This result can 
be seen complementary to Benaych-Georges and Nadakuditi (2012). They consider finite 



rank perturbations of a sequence of random matrices (Xm), where Xm is a M x TV- 
matrix. Under certain assumptions they show an almost sure convergence of the largest 
singular values in the limit M, Nm — > 00. Since we only make assumptions on the first 
two singular values and the first left singular vectors of the perturbation matrices (Cm), 
the limit limM-s.00 Ai (Cm + Em) does not exist in general. However, under the condition 
Ai — A2 S> 10<7 2 , there exists a small interval containing liminfM->oo Ai (Cm + Em) and 
lim sup^^^ Ai (Cm +Em) almost surely. Note that if Ai — A2 > 4<r, then the upper bound 
in Corollary 1 is already better than the obvious bound Ai + 2a on lim sup^/^^ Ai (Cm + 
E M ). 

Before we prove Corollary 1, let us give two examples of sequences (ui)i S K satisfying 
condition (15.111: 



• All but finitely many u,'s are zero. 

• The sequence is bounded and bounded away from zero. 

Proof of Corollary 1. Now we give a computation of (|5.2p . For this purpose we make a 
slight abuse of notations. We write C and E for the matrices Cm and Em- Further let 
(uii, vmi) be the right singular vector of Cm corresponding to Ai(Cm)- Consider the 
lower bound on liminf XACm + Em)' 

M->oo 



Xx{C + E) > ||7ri(C + S)|| S2 = ^Xi{Cf +2tT(C T ir 1 E) + \\mEW 



A l( C) 2 + 2A l( C)^ -gg£ E Tl+ £ ^p^E ik E jk . 
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By a SLLN of Thrum 1(1987)1 we get 

M 



E 



VnUji 



. i(E^li) 1 / 2 

Moreover, by the subsequent theorem follows 

M 

UnUjl 



E jt a 4- 0. 



So it holds 



A? + cr 2 < liminf Ai (Cm + #m) a.s. 

M->oc 



It remains to prove the upper bound. It holds 

A X (C + E) = ^j\\^{C + E)\\l 2 - Wn^C + E)\\l 2 + |MC + £)|| 



< 



\ 



M 



Il' + a 2 + A 1 (C) 2 + 2A 1 (C) J2 



VnUji 



fax 



-Ea. 



A I 



\ A 1 (C) 2 -A 2 (C)' 



r a 2 + ( T 2 + A 1 (C) 2 + 2A 1 (C) J2 



VnUji 



£—1 (spM 2 U/2 
4 . 7 = 1 l/_/i=l U il) 



E 



.)•■ 



By results of Bai, Krishnaiah, and Yinlfl988) and Bai, Silverstein, and Yin K 1988)1 we know 
that the fourth-moment condition on the entries of E is necessary and sufficient for the 
almost sure convergence of o~\ to 2c Applying this to the last line of the computation 
yields the desired claim. □ 

We close this article by a strong law of large numbers for empirical covariance matrices. 
In this result the empirical covariance matrix is considered as a quadratic form. 



Theorem 5 (SLLN for empirical covariance matrices) 

Let (ui)igM be a sequence of real numbers, such that there exist /3 > > with 
U \ < for all M e IN, where B = [(M* - lf\ . 



(5.3) 



Furthermore let (Eij)ij^ be a doubly indexed sequence of centered i.i.d. random variables 
with variance a 2 and finite fourth moment. By (Em) we denote the sequence of M x M 
random matrices l/VM(Eij)ij<M ■ Then we have 



u M E M E M u M c 2 , 



(5.4) 
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where 



M 



,u M y 



Proof. Set 

M M 



Z M ■= u M E M E M u M - M — - y~] UiUjE ik Ej 



Therefore 2m is the sum of Wm and 2Xm given by 

M M 



M M i-1 

Xm := TTv^f — 2 ZZ UiUjEikEjk. 
M l~*i=i u i fe=i i=i j=i 

First we show, that Wm converges to tr 2 almost surely. This part is an ada ption of some 
arguments of the classical strong law of large numbers (cf. Etemadi I (T 9811 ). Here we do 
not even need truncation arguments, since the entries Eij have finite fourth moments. By 
the Borel-Cantelli Lemma we get for fc„ := , n € IN, that (W kri ) converges to a 2 almost 
surely. By monotonicity of (M^ i=1 u 2 Wm), condition (|5.3[) and picking k n < M < k n +i, 
it follows 

a 2 < liminf Wm < limsup Wm < & 2 a.s. 

M-Hx> M-Hxi 

Now consider (Xm)- Let k n be as before. Then again by the Borel-Cantelli Lemma (X kri ) 
converges almost surely to 0. For any M € IN pick n € IN, so that k n < M < k n +i- We 
have 

_^ k n k n i—1 

\Xm\ < M 2 I u iUj E ik E jk \ 

M 2-ii=l U i k=l i=l j = l 

. k„ M i-1 

M 2^i=l u i k=li=k„ + lj=l 



M M i-1 

,2 



m ^m 2 1 Z EE^^^i 

J " 2_,i=l U i fc=fc„ + l i=l 3=1 



Clearly, the first and the last term go to zero almost surely. It remains to prove, that 

^ k„ M i-1 

Vm ■= M - J2 J2 u iUj E ik E jk a.s. 
iV1 2^i=i u i fe=i i=fe„+i j=i 
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Therefore we estimate Var(Vjvf): 

4 M h 

Var(V M )<— ^- -2 E u lrJ2 u i 



~2 "l 



< 



< 



4 v-vM 2 



a 4 



EiV-i 

< 



By the Borel-Cantelli Lemma and Chebyshev's inequality again we conclude the desired 
claim. □ 

If only finitely many entries it, are non-zero, then the almost sure convergence follows 
directly from the classical strong law of large numbers, since 



m I M 

Zm = 



1 M I M 



and the summands 




are i.i.d. for Mq large enough. 
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